On the Educational Validity of Research in Educational Technology

Alsop and Tompsett argued in 2007 that research in educational technology was over-focused on technological design and that empirical research that was pertinent to educational change should be assessed against two distinct scales. The first considers the quality of the experimental design in validating the conclusions that are reached, whilst the second assesses the extent to which the experiment reflects normal practice. They suggested that research should only claim to provide evidence for specific change in educational practice if it met the highest levels on both scales. This study, structured in the form of a systemic review covering more than 40 years of published research, provides the first systematic assessment of their model. This analysis suggests that, in at least one key domain in higher education, there is no current reliable research evidence, published in English, that supports the case for a change in practice based on the introduction of information and communication technology (ICT). In this context, practitioners would appear to be fully justified in considering innovations that do not depend on ICT. If practitioners are expected to change, then research must be designed so that the results are transferable, irrespective of the methodology used.